Papers with model testing
Why Does ChatGPT “Delve” So Much? Exploring the Sources of Lexical Overrepresentation in Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Scientific English is currently undergoing rapid change, with words like “delve,” “intricate,” and “underscore” appearing far more frequently than just a few years ago. |
| Approach: | They propose a formal method to characterize scientific English linguistic changes . they propose lexical overrepresentation by reinforcement learning from human feedback . |
| Outcome: | The proposed method yields 21 focal words whose increased occurrence in scientific abstracts is likely the result of LLM usage. |
Beyond Testers’ Biases: Guiding Model Testing with Knowledge Bases using LLMs (2023.findings-emnlp)
Copied to clipboard
Chenyang Yang, Rishabh Rustogi, Rachel Brower-Sinning, Grace Lewis, Christian Kaestner, Tongshuang Wu
| Challenge: | Identifying what to test is a step that is largely ignored and poorly supported. |
| Approach: | They propose an interactive tool that supports requirements elicitation for guiding model testing. |
| Outcome: | The proposed tool can help practitioners test models in real-world settings . |
ONEBench to Test Them All: Sample-Level Benchmarking Over Open-Ended Capabilities (2025.acl-long)
Copied to clipboard
| Challenge: | ONEBench enables custom benchmarks for specific capabilities while reusing and aggregating samples. |
| Approach: | They propose a new paradigm that consolidates individual evaluation datasets into a unified, ever-expanding sample pool. |
| Outcome: | The proposed model evaluation framework is based on dynamic, sample-level evaluation. |